Integrated multi-modality imaging system and method

ABSTRACT

An integrated, multi-modality imaging technique is disclosed. The embodiment described combines a split-magnet MRI system with a digital x-ray system. The two systems are employed together to generate images of a subject in accordance with their individual physics and imaging characteristics. The images may be displaced in real time, such as during a surgical intervention. The images may be registered with one another and combined to form a composite image in which tissues or objects difficult to image in one modality are visible. By appropriately selecting the position of an x-ray source and detector, and by programming a desired corresponding slice for MRI imaging, useful combined images may be obtained and displayed.

FIELD OF THE INVENTION

The present invention relates generally to the field of imaging systemssuch as those used in medical diagnostics. More particularly, theinvention relates to an integrated system that makes use of differentmodalities in a complementary fashion to permit feedback to surgeons andother medical professionals of physical conditions of a subject,particularly during interventionary procedures.

BACKGROUND OF THE INVENTION

A wide variety of imaging systems have been developed and are presentlyin use in the medical field. The systems may be generally categorized ina series of “modalities,” with each modality being characterized by itsparticular physics, control, utility, and so forth. For example,magnetic resonance imaging (MRI) systems are commonly employed forproducing images of gyromagnetic material within a subject of interest.Over recent years, such systems have become particularly refined inproducing high quality and reliable images of internal organs and otherparticular types of tissue, in various orientations within the subject.X-ray-based techniques have also grown considerably from their initialroots in analog systems utilizing photographic film. Modem x-ray-basedmodalities include digital x-ray systems which produce electronic datasets representative of picture elements or pixels within an array thatcan be reconstructed into a useful and high quality image. Otherx-ray-based techniques include computed tomography (CT) systems in whichx-ray radiation traverses a subject, impacts a detector, and resultingsignals are reconstructed by a computer into a useful image through thesubject. Still other modalities include positron emission tomography(PET), ultrasound, and so forth.

While the various modalities of imaging systems used in the medicalfield have improved dramatically in recent years, and continue toimprove, they have tended to develop in isolation. MRI systems, forexample, are typically used for specific purposes, such as imaging softtissues. X-ray-based modalities are often used in other situations forwhich MRI systems are less suitable. In such systems, where images aredesired of tissues or anatomies which cannot normally be identified orcontrasted from neighboring structures, various approaches may beemployed to provide the desired contrast, typically through the use ofliquid contrast agents which are injected into the patient prior to theexamination sequence. These contrast agents, however, do not necessarilyprovide the particular tissue identification desired, may not beretained for the time and in the locations desired for the entireprocedure, and may cause complications for certain patients. Othertechniques have been developed to attempt to identify probes, catheters,and the like, through the use of one or another modality system. Suchprobes, for example, may include coils which respond to the pulsesequences of MRI systems, to provide feedback to a surgeon during asurgical intervention such as catheterization, and so forth.

In certain procedures, it would be useful to provide additional feedbackto medical personnel of the state of tissues and anatomies based upon acombination of imaging modalities. For example, during catheterization,angioplasty, and similar procedures, MRI systems may permit a surgeon toidentify soft tissues through which a probe is inserted, but are notnecessarily well suited to imaging tissues indicative of the actuallocation of the probe. Because surgical interventions happen in realtime, currently available technologies for separate modality imaging aresimply ill suited to providing this type of information and feedback.There is a need, therefore, for an improved technique for supplyinganatomical images to medical professionals which overcomes thelimitations of separate modalities such as MRI and x-ray-based systems.

SUMMARY OF THE INVENTION

The present invention provides an integrated imaging system designed torespond to these needs. The technique may be applied as a combination ofvarious different imaging modalities, but is particularly well suited tocombining MRI systems with x-ray-based systems, such as digital x-rayfluoroscopy systems. The systems are combined in a complimentary andcooperative manner, such that real-time images may be produced of softtissues through use of MRI imaging sequences, while images of more denseor contrasting tissues and objects may be produced through the x-raysystem. The systems may be physically combined by positioning aspecially-adapted support structure for a digital x-ray. apparatus in anMRI system. Separate images may be produced by the two systems, with thedesired anatomy of soft tissues being projected on a screen associatedwith the MRI system, while the x-ray image is displayed on a separatescreen. Alternatively, the system may be adapted to register and combinethe images to provide real-time feedback of all of the structures ofinterest, thereby making use of the strength of the combined modalitiesin the resulting imaging. In addition to real-time imaging, the systemmay be employed to produce images which are registered or associatedwith one another, with the still images being available for viewing,storage, transmission, and so forth.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a diagrammatical representation of an integrated,multi-modality imaging system for use in producing images of internaltissues of a patient in a medical procedure;

FIG. 2 is a block diagram representing the principle structures of thesystems of the embodiment of FIG. 1;

FIG. 3 is a functional block diagram illustrating the various functionalsub-systems of the arrangement of FIG. 1 and FIG. 2 for use in producingboth separate and combined images; and,

FIG. 4 and FIG. 5 are exemplary images produced through use of theintegrated system in accordance with certain aspects of the presenttechnique, for imaging the location of a probe in a blood vessel of asubject.

DETAILED DESCRIPTION OF THE INVENTION

Turning now to the drawings, and referring first to FIG. 1, anintegrated, multi-modality imaging system 10 is illustrated as includinga data acquisition station 12 and a data processing and control station14. In the illustrated embodiment, the imaging system includescomponents of magnetic resonance imaging (MRI) and x-ray modalities.Specifically, the system includes a split magnet MRI system 16 and adigital x-ray system 18 configured to generate images during a medicalprocedure to provide feedback to a medical diagnostic or surgical team.It should be noted, however, that while the MRI and x-ray modalitiesdescribed herein are combined by way of example, various othermodalities may be combined in similar manners to draw upon the strengthsof the particular imaging modalities involved in viewing specifictissues, surgical devices, anatomical features and physiologicalfunctions.

In the arrangement illustrated in FIG. 1, MRI system 16 includes a coilhousing 20 which is divided into a left-hand section 22 and a right-handsection 24 separated by an opening or access region 26. Generallyperpendicular to opening 26, housing 20 forms a patient aperture 28designed to receive a table 30 on which a patient 32 may be positioned.The patient may thus be disposed at various locations within the coilhousing to orient desired portions of the patient's anatomy within theregion of opening 26 and to extend regions of the patients anatomybeyond an end of the housing, such as for access to the patient's legsor abdomen. MRI system 16 further includes an MRI controller, designatedgenerally by reference numeral 34 in FIG. 1 and a viewing screen 36positioned adjacent to the data acquisition station for displayingreconstructed images based upon the magnetic resonance imaged dataacquired via the scanner.

X-ray system 18 includes an x-ray source 38 positionable within oradjacent to opening 26 on one side of the patient, and a digitaldetector array 40 positionable on an opposite side of the patient. Thex-ray source and digital detector may be movable together on a fully orpartially rotatable gantry 42, such as for selecting an appropriateimaging orientation with respect to the patient. X-ray system 18 furtherincludes an x-ray system controller 44 for regulating operation of thex-ray source and detector, and for collecting and processing image dataduring operation. An x-ray image viewing screen 46 is provided adjacentto the data acquisition station 12 for displaying x-ray images to themedical diagnostic or surgical team.

An integrated system controller 48 is linked to MRI controller 34 and tox-ray system controller 44 to coordinate production of the desiredimages, as well as to perform combinations or composites ofmulti-modality images as described more fully below. Data processing andcontrol station 14 further includes components for facilitatinginterfacing with a radiologist, clinician, or a member of a surgicalteam, as indicated generally at reference numeral 50. Thus, operationstation 50 may include such peripheral devices as a computer monitor 52,and input devices such as a standard computer keyboard 54 and mouse 56.In addition to the components illustrated diagramatically in FIG. 1, thesystem may further include communications components for transmittingimages generated by the data acquisition station to remote locations,such as for teleradiology techniques, and for storing or archivingimages, such as picture archiving and communication systems.

FIG. 2 represents certain of the operational components of the MRI andx-ray systems in somewhat greater detail. As will be appreciated bythose skilled in the art, MRI system 16 includes a series of coils whichcan be precisely controlled to generate desired magnetic fields andradio frequency pulses to produce and sense magnetic resonance emissionsfrom gyromagnetic, material within the patient anatomy. These coilsinclude a primary field coil 58 which generates a uniform magnetic fieldgenerally aligned with patient bore 28. Gradient field coils 60, 62 and64 are provided for generating magnetic gradient fields generallyorthagonally oriented with respect to one another. A radio frequencycoil 66 is provided for generating pulsed radio frequency signals inresponse to which the gyromagnetic material will produce the magneticresonance emissions. In particular, the coils of system 16 arecontrolled by external circuitry to generate desired fields and pulses,and to read emissions from the gyromagnetic material in a controlledmanner.

As will be appreciated by those skilled in the art, when the material,typically bound in tissues of the patient, is subjected to the primaryfield, individual magnetic moments of the paramagnetic nuclei in thetissue attempt to aligned with the field but precess in a random orderat their characteristic or Larmor frequency. While a net magnetic momentis produced in the direction of the polarizing field, the randomlyoriented components of the moment in a perpendicular plane generallycancel one another. During an examination sequence, an RF frequencypulse is generated at or near the Larmor frequency of the material ofinterest, resulting in rotation of the net aligned moment to produce anet transverse magnetic moment. Radio signals are emitted following thetermination of the excitation signals. This magnetic resonance signal isdetected in the scanner and processed for reconstruction of the desiredimage.

Gradient coils 60, 62 and 64 serve to generate precisely controlledmagnetic fields, the strength of which vary over a predefined field ofview, typically with positive and negative polarity. When each coil isenergized with known electric current, the resulting magnetic fieldgradient is superimposed over the primary field and produces a linearvariation in the overall magnetic field strength across the field ofview. Combinations of such fields, orthagonally disposed with respect toone another, enable the creation of a linear gradient in any directionby vector addition of the individual gradient fields.

The gradient fields may be considered to be oriented both in physicalplanes, as well as in logical axes. In the physical sense, the fieldsare mutually orthagonally oriented to form a coordinate system which canbe rotated by appropriate manipulation of the pulsed current applied tothe individual field coils. In a logical sense, the coordinate systemdefines gradients which are typically referred to as slice selectgradients, frequency encoding gradients, and phase encoding gradients.

The slice select gradient determines a slab of tissue or anatomy to beimaged in the patient. The slice select gradient field may thus beapplied simultaneous with a selective RF pulse to excite a known volumeof spins within a desired slice that precess at the same frequency. Theslice thickness is determined by the bandwidth of the RF pulse and thegradient strength across the field of view.

A second logical gradient axis, the frequency encoding gradient axis isalso known as the readout gradient axis, and is applied in a directionperpendicular to the slice select gradient. In general, the frequencyencoding gradient is applied before and during the formation of the MRecho signal resulting from the RF excitation. Spins of the gyromagneticmaterial under the influence of this gradient are frequency encodedaccording to their spatial position across the gradient field. ByFourier transformation, acquired signals may be analyzed to identifytheir location in the selected slice by virtue of the frequencyencoding.

Finally, the phase encode gradient is generally applied in a sequencebefore the readout gradient and after the slice select gradient.Localization of spins in the gyromagnetic material in the phase encodedirection is accomplished by sequentially inducing variations in phaseof the precessing protons of the material by using slightly differentgradient amplitudes that are sequentially applied during the dataacquisition sequence. Phase variations are thus linearly imposed acrossthe field of view and spatial position within the slice is encoded bythe polarity and the degree of phase difference accumulated relative toa null position. The phase encode gradient permits phase differences tobe created among the spins of the material in accordance with theirposition in the phase encode direction.

As will be appreciated by those skilled in the art, a great number ofvariations may be devised for pulse sequences employing the logical axesdescribed above. Moreover, adaptations in the pulse sequences may bemade to appropriately orient both the selected slice and the frequencyand phase encoding to excite the desired material and to acquireresulting MR signals for processing.

X-ray system 18 provides imaging of anatomies which are less suitablefor MRI imaging, including images of bone, external probes, catheters,and so forth. X-ray source 38 will typically include a collimator whichpermits a stream of radiation to pass into opening 26 in which thepatient is positioned. A portion of the radiation passes through andaround the subject and impacts digital x-ray detector 40. Detector 40converts the x-ray photons received on its surface to electrical signalswhich are acquired and processed to reconstruct an image of the featureswithin the subject patient.

In a presently preferred embodiment, detector 40 consists of ascintillator that converts the x-ray photons received on the detectorsurface during examinations to lower energy (light) photons. An array ofphoto detectors then converts the light photons to electrical signalswhich are representative of the number of photons or intensity of theradiation impacting individual pixel regions of the detector regions.Readout electronics convert the resulting analog signals to digitalvalues that are processed, stored, and displayed as reconstructed imageson display screen 36. The array of photo detectors may be made of asingle piece of amorphous silicon. The array elements are organized inrows and columns with each element consisting of a photo diode and athin film transistor. The cathode of each diode is connected to thesource of the transistor, and the anodes of all diodes are connected toa negative bias voltage. The gates of the transistors in each row areconnected together and the row electrodes are connected to scanningelectronics for reading the analog signals produced upon receipt ofradiation. The drains of the transistors in a column are connectedtogether and an electrode for each column is connected to readoutelectronics. The detector permits images to be produced by sequentiallyenabling rows of a detector and reading out the signals corresponding tothe individual picture element or pixel regions.

In the illustrated embodiment, MRI system 16 and x-ray system 18 arecoupled to their respective controllers 34 and 44, and their operationis coordinated through integrated system controller 48. Systemcontroller 48 may be located in the immediate vicinity of the dataacquisition station, typically in or near a surgical ward. The systemmay transmit images in real time or from memory to PACS andteleradiology stations, as indicated at reference numeral 70 in FIG. 2.Moreover, system controller 48 may combined images generated through theMRI and x-ray modalities, and by other modalities where desired, toutilize the strengths of both modalities. In the embodiment of FIG. 2,such combined images may be displayed on a single viewing screen asindicated at reference numeral 68.

As will be appreciated by those skilled in the art, MRI systems areparticularly well suited to imaging certain soft tissues comprisinggyromagnetic or paramagnetic molecules. X-ray systems, on the otherhand, are particularly well suited to imaging other types of materialshaving densities or absorption properties which provide contrast whenviewed in the x-ray radiation band. By generating both MRI and x-rayimages in real time, or in sufficiently closely spaced sequentialsampling periods to provide useful feedback to surgical teams, thesurgical teams may continuously monitor the location of tissues andinterventional tools during a surgical procedure. Moreover, by orientingthe source 38 and detector 40 of the x-ray system in a position which iscomplementary to slices generated by MRI system 16, projections may begenerated by integrated system controller 48 which include registeredimages from both modalities combined to illustrate the position ofvarious tissues or of interventional tools and the like within theimaged tissues.

Functional circuitry for the control of the MRI and x-ray components ofsystem 10 and for processing resulting image data is representeddiagrammatically in FIG. 3. As shown in FIG. 3, MRI controller 34includes a control circuit 72 which may itself include a centralprocessing unit or digital signal processor of a general purpose orapplication-specific computer or work station. Control circuit 72implements programming code for executing specific pulse sequences inthe commanded MRI examinations and produces the pulse sequences viaamplifier and drive circuitry 74 and transmission and receive circuitry76. In general, circuitry 74 includes amplification electronics forconverting the drive commands for gradient coils 60, 62 and 64 toelectrical pulses which generate the desired magnetic fields. Similarcommand signals are applied by control circuit 72 to circuitry 76 todrive RF coil 66. In the illustrated embodiment, circuitry 76 includesboth transmission and receive electronics for amplifying the RF commandsignals in an active mode, and for receiving resulting magneticresonance signals in a passive mode. The signals are applied bycircuitry 76 to control circuit 72 for processing. Control circuit 72 isalso coupled to memory circuitry 78, such as volatile and non-volatilememory, for storing pulse sequence descriptions, examination protocols,configuration parameters, image data, and so forth. Interface circuitry80 is provided for communicating examination requests, configurationparameters, and image data between MRI controller 34 and integratedsystem controller 48.

X-ray system controller 44 also includes a control circuit 84 which,similarly, may include a central processing unit or digital signalprocessor of a conventional computer or work station. Control circuit 84is coupled to generator circuitry 86 which controls the production ofx-ray radiation at source 38. As will be appreciated by those skilled inthe art, generator 86 commands electrical discharges within an x-raytube to produce a stream of x-ray radiation upon onset of an x-rayexamination. Control circuit 84 is also coupled to detector interfacecircuitry 88 which serves to receive and process signals from detector40. Detector interface circuitry 88 is configured to originate timingand control commands for row and column drivers of the detector and toprocess resulting signals sampled from the detector. The detectorcontrol circuitry 88, and control circuit 84 execute various signalprocessing and filtration functions, such as for adjustment of dynamicranges, interleaving of digital image data, and so forth. The circuitrythus commands operation of the x-ray imaging system to executeexamination protocols and to process the acquired image data. Theexamination protocols carried out by the circuitry will be defined incode stored in memory circuit 90. Memory circuit 90 may also serve tostore system configuration parameters and image data, both raw andprocessed. An interface circuit 92 is, provided for exchanging suchparameter configuration data and image data between control circuit 44and integrated system controller 48.

Both control circuitry 34 and 44 may include separate operator workstations for regulating their functions independent of integrated systemcontroller 48. In the embodiment illustrated in FIG. 3, therefore,operator stations 82 and 84 are provided in controllers 34 and 44,respectively. Where desired, functionality for operator interface may becombined into one single work station, such as a work station 102associated with integrated system controller 48.

Integrated system controller 48 includes a control circuit 96 which,again, may comprise a standard central processing unit or digital signalprocessor of a general purpose or application-specific computer. Codeexecuted by control circuit 96 is stored in memory circuitry 100 whichmay comprise both volatile and non-volatile memory, tape drives, opticalstorage devices, and so forth. Interface circuitry 98 is networked withinterface circuitry 80 and 92 of the individual system controllers, topermit the exchange of examination requests and configurations, and thetransfer of both raw and processed image data. Operator station 102permits the surgical team to configure and regulate the operation ofcontroller 48, and to process images acquired during procedures.

As described above, images produced and processed by MRI and x-raysystems 16 and 18 may be displayed separately or combined to provideregistered feedback of various tissues, instruments, or features ofinterest. FIGS. 4 and 5 represent such registered or combined imagesproduced by both systems. In FIG. 4, an image 104 of tissue 106, and ablood vessel 108, is produced by scaling an overlaying of both MRI andx-ray data. The data may be acquired at any convenient orientation, suchas to provide a plan view of the blood vessel as a probe or catheter 110is advanced during a surgical procedure. As shown in FIG. 5, a similarimage 112 may be generated by re-orientation of the source and detectorof the x-ray system and by selecting a correspondingly oriented imagingslice for the MRI system, to produce a similar image, but in a differentprojection to permit a surgical team to follow the advance of the probe110.

The foregoing technique, combining MRI and fluoroscopy imagingtechnologies, may be employed in a number of procedures andapplications. For example, as illustrated in FIGS. 4 and 5, thetechnique may be used to image blood vessels and advancing catheters soas to avoid the need for contrast agents employed in conventionalprocedures. Similarly, the arrangement may be employed in angioplastyprocedures, such as to track the deployment or positioning of a stentand to image the degree to which probes, catheters, or stints influencethe flow of blood visible via magnetic resonance imaging. The techniquemay also be applied for application of specific drugs or treatments tospecific locations, such as in regions of the brain or for treatment oftumors. In such cases, high atomic weight materials may be employed toabsorb x-rays but to avoid generating induced currents which may affectthe magnetic resonance images.

Although in the foregoing description, reference has been made to a useof the integrated imaging system in various surgical procedures, itshould be noted that the system may be employed for imaging purposesalone. Thus, by, combination of different imaging modalities such as MRIand x-ray systems, a patient may be positioned once for creation ofseparate or composite images via both modalities. Again, the particularimage and image orientation may be selected by appropriately positioningthe x-ray source and detector, and by selecting a corresponding slicefor MRI imaging.

1-29. (canceled)
 30. A method for performing an invasive procedure usingone or more multi-modal reference images, the method comprising thesteps of: (a) coordinating the operation of a first imaging modality anda second imaging modality; (b) generating a first image data set fromthe first imaging modality substantially simultaneous with generating asecond image data set from the second imaging modality; (c)reconstructing an image of a region of interest based upon the firstimage data set and the second image data set; (d) displaying the image;and (e) performing an invasive procedure on the region of interest basedupon the displayed image.
 31. The method as recited in claim 30, whereinsteps (a) through (c) are repeated to generate one or more subsequentimages of the region of interest.
 32. The method as recited in claim 31,further comprising displaying the one or more subsequent images.
 33. Themethod as recited in claim 32, wherein displaying the one or moresubsequent images comprises simultaneously displaying the image and theone or more subsequent images.
 34. The method as recited in claim 30,wherein one of the first imaging modality and the second imagingmodality comprises a digital X-ray imaging system.
 35. The method asrecited in claim 34, wherein the digital X-ray imaging system comprisesa fluoroscopic X-ray imaging system.
 36. The method as recited in claim30, wherein one of the first imaging modality and the second imagingmodality comprises a magnetic resonance imaging system.
 37. The methodas recited in claim 36, wherein the magnetic resonance imaging systemcomprises a split housing.
 38. method as recited in claim 37, whereinthe split housing is one of a vertically split housing and ahorizontally split housing.
 39. The method as recited in claim 30,wherein one of the first imaging modality and the second imagingmodality comprises a tomographic imaging system.
 40. The method asrecited in claim 30, wherein one of the first imaging modality and thesecond imaging modality comprises an ultrasound imaging system.
 41. Themethod as recited in claim 30, wherein one of the first imaging modalityand the second imaging modality comprises a tomosynthesis imagingsystem.
 42. A method for performing an invasive procedure using one ormore multi-modal reference images, the method comprising the steps of:(a) coordinating the operation of a first imaging modality and a secondimaging modality; (b) generating a first image data set from the firstimaging modality substantially simultaneous with generating a secondimage data set from the second imaging modality; (c) reconstructing animage of a region of interest based upon the first image data set andthe second image data set; (d) displaying the image during an invasivesurgical procedure; and wherein steps (a) through (c) are repeated togenerate one or more subsequent images of the region of interest.
 43. Amulti-modality imaging system comprising: means for coordinating theoperation of a first imaging modality and a second imaging modality;means for generating a first image data set from the first imagingmodality substantially simultaneous with generating a second image dataset from the second imaging modality; means for reconstructing an imageof a region of interest based upon the first image data set and thesecond image data set; and means for displaying the image to one or moremedical personnel performing an invasive procedure on the region ofinterest.